Anaphora resolution: a multy-strategy approach

Anaphora resolution has proven to be a very difficult problem; it requires the integrated application of syntactic, semantic, and pragmatic knowledge. This paper examines the hypothesis that instead of attempting to construct a monolithic method for resolving anaphora, the combination of multiple strategies, each exploiting a different knowledge source, proves more effective, theoretically and computationally. Cognitive plausibility is established in that human judgements of the optimal anaphoric referent accord with those of the strategy-based method, and human inability to determine a unique referent corresponds to the cases where different strategies offer conflicting candidates for the anaphoric referent. 1, I n t r o d u c t i o n : T h e C o m p l e x i t y o f A n a p h o r a R e s o l u t i o n Anaphora is a pervasive phenomenon in natural language communication, whether it be complex multi-party human discourse or more constrained bilateral human-computer dialog. Whereas the ubiquity of the phenomenon and tire necessity to cope with it in the construction of friendly natural language interfaces has long been recognized [13, 8, 9, 17, 15], no truly comprehensive computational approaches for anaphora resolution have been proposed. The RUS parser [2], the XCALIBUR system [7], and other operational natural language systems implement very rudimentary methods. And, theoretical work in linguistics is primarily concerned with certain types of syntactic intrasentential coreference, rather than pragmatic intersentential anaphora. AnaPhors typically refer back to other constituents in the same sentence, or to constituents in earlier utterances in fire discourse. Syntactic information plays a central role in establishing appropriate referents for the former case, intrasentential anaphora [17]. But, semantic and pragmatic infonnation is absolutely required in the latter case, intersentential anaphora [15, 9]. This paper addresses the problem of intersentential anaphora resolution, integrating caseframe semantics [10, 12, 5] and more global dialog coherence structures [11, 15, 14]. Empirical studies have shown that intersentential anaphora I is far more frequent and more crucial in designing interactive natural language interfaces 2 [8]. 96 2. T h e P r o b l e m : S e m a n t i c s a n d P r a g m a t i c s D o m i n a t e Finding the appropriate anaphoric referent has been long recognized as a difficult problem, requiring lnuch ~emautic and pragmatic lmowledge. Consider, for instance, the tbllowing two sets of examples: John took the cake from the table and ate it. John took the cake from the table and washed it. Tile robot pushed the box towards the conveyor belt. But, it goojSd and dropped it on its way there. Semantic preference constraints (e.g., [18, 11), if pmp:~rly coded, suffice to resolve the first example. The pt'eferre'l object of ingestation is an edible substance. It is a little more difficult to mechanize a process that excludes things such as cakes from being the object of washing. One cannot simply write a "NOT(edibley restriction on the object case of the verb "to wash". Alter ,all, vegetables and fruits are occasionally washed prior to eating them. Peltlaps a combination of typicality judgements with pragmatic knowledge exla'apolating the effects of attempting to drown a cake in sink full of water comes into play. Or, more abstract irffemntial constraints are appropriate, snch as requiring that the object of wash be unchanged by immersion in water. Interestingly, Subjects given only the "...and washed it" sentence report consistently that they didn't even consider the cake a reasonable referent for "it". In the robot example, there ate four anaphoric referents, counting the possessive "its" and the locative "there", referring to three different antecedents. Although subjects report little difficulty ascertaining the referent for each anaphor in a consistent manner, it appears that sophisticated semantic~ are required. Why is the referent for "it" in "it goofed and dropped..." the robot rather than the box or the conveyor belt? One could argue that the box cannot take action, but what allows a robot to goof and not a conveyor belt? Is it something as subtle as the degree to which the former can be anthropomolphized being greater than the degree to which the latter can be anthmpomorphized? The difficulty in anaphoric referent specification in narratives has been argued convincingly by many researchers including Chamiak in his work on children's story comprehension [9], where substantial pragmatic domain knowledge must be brought to bear, and by one of the authors [4], where knowledge of goals and personality traits is required to resolve difficult referents. Hence, the hypothesis that anaphor resolution in its fnll generality is at best a diffictflt problem, and at worst an almost intractable one, is well supported. Nevertheless, somewhat less ambitious endeavors can prove far more tractable, and yet be of major practical Import. Hayes [13] argued for the notion of limitod-domain anaphora in a natural language interface to an electronic mail system. Webber [17] demonstrated that intrasentential anaphora was more tractable than its intersentential counterpart, largely through the categorization of syntactic devices absent from larger textual or dialog segments. This paper explores an intemrediate position: addressing much larger classes of anaphors than those of Hayes [13] in a systematic mariner, but stopping short of full generality, which requires unbounded pragmatic knowledge and inference. We explore the central hypothesis that anaphora resolution may be best accomplished through fire combination of a set of strategies, rather than by a single monolithic method. The apparent complexities lie in the combination of these multiple strategies to produce syntactically, semantically arid pragmatically sound anaphoric resolutions. In the multiple examples analyzed, 3 unambiguous resolutions reported by human subjects correspond to situations where the applicable strategies concur on the referont of an anaphor, and disagreement on the con'eet referent by the human subjects corresponds to situations where the applicable strategies propose different candidate referents for file anaphor in questitm. 3. Multiple Resolution Strategies In this section we propose a general framework for anaphor resolution ba~d on the integration of multiple knowledge sources: sentential syntax, case-frame semantics, dialog structure, and general world knowledge. The underlying theoretical tenet is: Anaphor resolution is not a monolithic autonomous process; it requires access and integration of all the knowledge sources necessary for dialog and text lnterprel'ation. These linguistic knowledge sources are brought to bear as constraints or preferences encoded as multiple resolntion strategies. Each source of knowledge usethl in resolving intersentential anaphnra is presented below, along with corresponding examples, and a statement of the anaphoric resolution strategy. 3.1. Local A naphor Constraints Certain anaphot~ carry with them constraints (number, gender, case, etc.) which must be satisfied by the candidate referents. For instant'e, gender uniquely specifies the anaphor in: John at~d Mary went shopping. He bought a steak. [he=John] Tile strategy here is trivial: Eliminate from consideration all candidate referents that violate the local constraints of the anaphor its question. A variant of this strategy has been implemented in RUS and in XCALIBUR. 3.2. Case-role Semantic Constraints Here the ease-role semantics impose constraints on what can fill them. If they are filled by an anaphor (which specifies few if any semantic features), the case role constraints must be also satisfied by the referent of tile anaphor, thus eliminating from consideration all candidate anaphor referents that violate constraints on the case role occupied by the anaphor. Consider our previous example, where the semantic constraints on the object case of "to eat" and "to wash" impose restrictions on the possible case fillem and prove sufficient to select a unique referent. John took the cake from the table and ate it. lit:cake] John took the cake from the table and washed it. lit=table] The slrategy here is also fairly simple: Eliminm'e from consideration all candidate referents that violate any case-constraint imposed on the anaphor its question~ Prefer those candidates that accord with typical ease fillers, in the absence of hard constraints. XCALIBUR implements this strategy directly though use of its caseframe grammar. With the I-rule mechanism, it was possible to implement an ad-hoc variant of this strategy in RUS as well. 3.3. Preconttition/Postcondition Constraints Using real-world knowledge and pragmatics, it is possible to say that a candidate antecedent cannot be the referent of an anaphor because some action occurring betwee n the referent and the anaphor invalidates the assumption that they denote one and the same object or event. John gave Tom an apple. He ate the apple. [he=Tom] Here, "he" refers to Tom, as Jolm no longer has the apple. The postcondition on give is that the actor no longer have the object being given, which ,:onfiicts with the precondition on eat that the actor have the item being eaten, if the actor is assumed to be John. The strategy is simple, but requires a fairly large amount of knowledge to be useful for a broad range of cases: Eliminate from consideration all candidate referents associated with actions whose postconditions violate the preconditions of the action containing the anaphor. 3.4. Case.role Persistence Preference We observe a pervasive form of "linguistic inertia" that manifests as a preference to assign the referent of an anaphor to the linguistic entity in the discourse context that filled the corresponding semantic case role in an earlier utterance. This is a generalized form of easerole parallelism, which has proven crucial in ellipsis resolution [8, 7, 5], although in anaphora resolution it is demoted from the status of a categorical constraint to that of a preference. Mary gave an apple to Susan. John also gave her an orange. [her=Susan] Mary gave an apple to Susan. She also gave John an orange. [she=Mary] The first anaphor relers to Susan, whereas the second anaphor refers to Mary. Clearly it is not a matter of primacy or recency, as the sentence structures are identical. Rather it is a case of structural parallelism. And, the semantic structnre dominates over the syntactic one. For instance, in the first example, "Susan" is the object of the "to" prepositional phrase, whereas the corelerent anaphor is in the indirect ol~iect position: two different syntactic roles that map into the same semantic case, recipient. In the second example above, both syntactic mad semantic structures coincide, and therefore the preference is stronger. Note, moreover, that the subject or directobject form of the pronoun ("she" vs "her") is not the primary source of discriminant knowledge. For instance, in the example below, one has only the anaphor "it", but the same referent discrimination occurs by semantic case-role parallelism: The robot gave the dog a bone. John also gave it some water, lit=dog] The robot gave the dog a bone. It also gave John some water, lit=robot] To provide more ammunition in support of semantic case role persistence, consider the following final example, with three possible referents to the anapher "him". It is clear that "Peter" is the preferred referent, once again due to the persistence of the underlying semantic recipient case. John carried the box of papers from Bill to Peter. He also sent him Mary's books. [he=John, him=Peter] The semantic preference strategy can be stated as follows: Search first for acceptable referents in the antecedent phrase (or phrases) that occur in the same semantic case role as the attaphor, l f a match satisfying all constraints is found, look no further; else search the other case roles. To our knowledge, this preference strategy has been neither proposed nor implemented prior to our work on the Universal Parser (reported below), yet it counts for a large number of anaphor resolutions in our sample set. 3.5. Semantic Alignment Preference A form of pragmatic "Occam's razor" exists in not postulating extra roles for the same objects in different sentences in the discourse. This preference is a more gcner~d and looser form of case role inertia, discussed above, in that the we have inmtia of the underlying action. For instance, in the example below, this preference manifests as preferring all departures to be from the park, and all arrivals m be at the club: Mary drove from the park to the club. Peter went there too. [there=chth] Mary drove from the park to the chub. Peter left there too. [there=park] The locative anaphor "there" refers to "the club" in the first example above, but refers to "the park" in the second example, yet both sentences share the identical syntactic structure and the same basic underlying semantic case structure. However, discourse cohesion